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Recently, it has been shown that the communities in neuronal networks of the integrate-and- 
fire type can be identified by considering patterns containing the beginning times for each cell to 
receive the first non-zero activation. The received activity was integrated in order to facilitate 
the spiking of each neuron and to constrain the activation inside the communities, but no time 
decay of such activation was considered. The present article shows that, by taking into account 
exponential decays of the stored activation, it is possible to identify the communities also in terms 
of the patterns of activation along the initial steps of the transient dynamics. The potential of this 
method is illustrated with respect to complex neuronal networks involving four communities, each 
of a different type (Erdos-Reny, Barabasi-Albert, Watts-Strogatz as well as a simple geographical 
model). Though the consideration of activation decay has been found to enhance the communities 
separation, too intense decays tend to yield less discrimination. 

PACS numbers: 87.18.Sn, 05.40Fb, 89.70.Hj, 89. 75. He, 89.75.Kd 



'Zora 's secret lies in the way your gaze runs over pat- 
terns following one another as in a musical score...' (I. 
Calvino, Inivisible Cities) 



I. INTRODUCTION 

Neuronal networks (e.g. [E S 13]) and complex net- 
works (e.g. 0, i, 1 0) can be understood as sister re- 
search areas. However, as the latter is much younger 
(especially regarding the developments from 1999), these 
two sisters have yet to get fully acquainted one an- 
other. Such a natural integration has already begun 
(e.g. [1, i, 0, HH, E E (ll d, Ei) and is poised 
to continue to the point that these two areas become 
not only close relatives, but also best friends. This in- 
tegration is particularly interesting for both neuronal 
networks and complex networks because of the comple- 
mentation of the approaches which have been respec- 
tively adopted. More specifically, while neuronal net- 
works have relied strongly on pattern recognition and 
dynamical systems, complex networks have been strongly 
focusing on structure, with a recent surge of interest 
on dynamics (e.g. 0, [13 )• However, as special em- 
phasis has been placed on the important problem of 
linear synchronization (e.g. (l7j). few works have ad- 
dressed non-linear or transient dynamics (e.g. 
In complex networks, emphasis has been placed on the 
modularity of the connections or community structure 
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(e.g. |20|, [211, \M [231, M l25|, M, l27|, IM l29|, |3C 

has important implications for both the structure and dy- 
namics of networks. The integration between neuronal 
networks and complex networks is henceforth referred 
to as complex neuronal networks, which has special im- 
portance for non-linear dynamical systems underlain by 
structured and complex connectivity. 

Recently [s^ [ssj , complex neuronal networks involving 
simple integrate-and-fire neurons (each neuron is repre- 



sented as a node) have been studied with respect to their 
transient dynamics. Figure [1] illustrates the type of neu- 
ronal cell adopted in those works. The incoming activa- 
tion, received through the n{i) dendrites, is integrated 
and accumulated in the internal state S{i) until its value 
exceeds the threshold T{i), in which case the cell fires, 
liberating the accumulated activation between the m(i) 
outgoing edges (axons). In the previous works [H,!!!], in 
order to maintain the total received activation, which was 
fed through a single selected neuron, the accumulated ac- 
tivation S{i) was uniformly distributed among the m{i) 
outgoing connections, each therefore receiving a share of 
S{i)/kout{i)) where koUt{i) = m{i) is the out-degree of 
node i. 




FIG. 1: The integrate-and-fire neuronal cell adopted in the 
previous works [sJiIHI incorporates three stages: (i) integrat- 
ing of input activations; (ii) memory of activation S{i); and 
(iii) non-linear transfer function involving a threshold T{i) 
(a hard limitter). While those previous works adopted full 
conservation of the activation (i.e. decay rate a = 0), in the 
present work the stored activation undergoes exponential time 
decay with rate a. 



Several interesting dynamic features are implied by 
such a simple neuronal model. First, the accumulation 
of the received activity is related to the important phe- 
nomenon of facilitation of firing. Roughly speaking, the 
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income of a spike into a cell enhances the probability of 
its future spiking by occasion of subsequent activations. 
Second, the non-linear element implies the activation to 
remain stored until the threshold is reached, which con- 
tributes strongly to constraining the activation locally in 
the network along topology and time. As all neurons are 
henceforth assumed to have the same threshold T = 1 (a 
biologically reasonable choice) , the distribution of outgo- 
ing activation implied by each spiking becomes impera- 
tive in order not to yield one spike at every time step. 
Similar effects can be obtained by associating weights 
smaller or equal to one to each edge (synaptic weight). 
The combination of such non-linear effects has been ob- 
served [3^ to contribute decisively for constraining, along 
a transient period of time, the activation inside the com- 
munity which contains the source of activation. Such an 
effect allows the identification of neuronal communities 
by considering the transient non-linear dynamics in the 
whole network while it is stimulated by sources of acti- 
vations placed at each of its neurons. It has been exper- 
imentally verified that the time it takes for each cell to 
receive non-zero activation in any of its dendrites, called 
the beginning activation time of each cell, seems to be 
particularly relevant for the identification of the commu- 
nities. Promising results were obtained with respect to 
two synthetic (networks including 3 and 4 communities 
with uniform connectivity) as well as a real-world net- 
work (C elegans [20j'). 

However, the previous investigations reported in [ssj 
considered no time decay of the stored activation S'(i), 
which seems to have been responsible for making the be- 
ginning activation times decisive for the proper identifica- 
tion of the communities. In the present work we consider 
the more biologically realistic situation involving expo- 
nential decays of the activations. More specifically, at 
each time step each stored activation is decreased at a 
constant rate a, i.e. 

S'+\t) = S\i) ~ aS\t) (1) 

where t is the time step and < a < 1. 

The net effect of the decay is to generally delay the 
firing of cells. Interestingly, such an effect seems to al- 
low proper identification of the communities also by con- 
sidering the average activation of the network along an 
interval of the transient dynamics, instead of only the be- 
ginning activation time. This is possibly a consequence of 
the fact of the enlarged period of time required to convey 
the activation from one community to another, which is 
enhanced by the decays. This possibility is experimen- 
tally investigated in the current article by considering 
hybrid networks containing four communities of differ- 
ent types (Erdo-Renyi, Barabasi- Albert, Watts-Strogatz 
as well as a simple geographical model). Several combi- 
nations of inter and intra-community intensities of con- 
nections are considered. The activation is averaged from 
the beginning of the source operation for a total of H 
steps along the transient dynamics. Then, the statis- 



tical method known as Principal Component Analysis 
(PCA) is applied to the activation patterns in order to 
reduce their dimensionality, which is optimally obtained 
by decorrelation of the activation. The original commu- 
nities could be properly detected in most cases, even for 
the Barabasi- Albert and geographical models. Combined 
with the investigations reported previously ^3§\, the re- 
sults obtained in the current work substantiate further 
the importance of the transient regime for characteriza- 
tion of modularity regarding both structure and dynam- 
ics in complex systems. With respect to the specific area 
of neuronal networks, the relationships between struc- 
tured connectivity, in the form of communities, and the 
activation and spiking dynamics provide several implica- 
tions for synchronization, pattern recognition and mem- 
ory. The proposed methodologies may also prove useful 
as practical methods for identification of communities in 
more general types of networks. 

The current article starts by presenting the basic con- 
cepts in complex neuronal networks, the four adopted 
theoretical models of complex networks, and the statis- 
tical method of Principal Component Analysis. The re- 
sults, discussion, and perspectives for future works are 
presented subsequently. 



II. BASIC CONCEPTS 

A directed, unweighted network F can be completely 
specified in terms of its adjacency matrix K. Each edge 
extending from node i to node j is represented K{j, i) = 
1. The absence of connection between nodes i and j 
implies K{j, i) = 0. The nodes which receive a direct 
edge from a node i are called the immediate neighbors of 
i. The out- degree of a node i is equal to the number of 
its immediate neighbors. 

Four theoretical models of complex networks (e.g. [1, 
S B 0) have been used in order to construct the hy- 
brid community networks considered in this work: Erdos- 
Renyi (ER), Barabasi- Albert (BA), Watts-Strogatz (WS) 
as well as a simple geographical type of network (GG). 
An Erdo-Renyi network (see also [3J|) can be obtained 
by establishing connections between pairs of nodes with 
constant probability. The BA networks were obtained 
by starting with mO nodes and progressively incorporat- 
ing new nodes with m edges, which are attached to the 
remainder nodes with probability proportional to their 
respective degrees. The WS structures were obtained 
by starting with a linear regular network of suitable de- 
gree and subsequently rewiring 10% of its edges. The 
geographical structures are obtained by distributing N 
nodes along a two-dimensional space and then connect- 
ing each pair of nodes whose distance does not exceed 
a given threshold. Though all these networks are undi- 
rected, we obtained the respective directed neuronal com- 
plex networks by considering the incoming on outgoing 
directions of each edge as dendrites and axons, respec- 
tively. Therefore, the so-obtained networks are directed 
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and have in-degree identical to the out-degree. 

The integrate- and- fire neuron adopted in this work has 
been described and discussed in the Introduction. The 
activation and spiking of all neurons in the network can 
be represented in terms of diagrams which are henceforth 
called activogram and spikegram, respectively. These di- 
agrams are matrices storing the transient activation or 
occurrence of spikes for every node. In this article, the 
activation of the network is always performed by injecting 
external activation of intensity 1 at each of the neurons. 
The time it takes for each neuron i, from the onset of the 
external initiation, to receive the first non-zero input is 
henceforth called its respective beginning activation time 
Ta{i, v). The time it takes for that neuron to produce the 
first spike is the beginning spiking time Ts(i,v). 

Because the activation patterns obtained with the 
source in each of the N neurons involve N measurements, 
a highly dimensional space is implied. As a consequence 
of the intrinsic correlations between the activation pat- 
terns, it is possible to apply the PCA method to opti- 
mally decorrelated those patterns and yield meaningful 
2D and 3D projections. Let each of the N observations 
V — {1,2,..., N} be characterized by the average acti- 
vations of all nodes as a consequence of the activation 
source placed at node v. These measurements can be or- 
ganized into respective feature vectors fy, with elements 
fv(i), i G {1, 2, . . . , TV}. Let the covariance matrix be- 
tween each pair of measurements i and j be defined as 

1 ^ 

where Hi is the average of fy{i) considering all the N 
observations (i.e. activations). The eigenvalues of C, 
sorted in decreasing order, are henceforth represented 
as Xi, i = 1,2,...,M, with respective eigenvectors vl. 
The matrix G given in Equation [3l obtained from the 
eigenvectors of the covariance matrix, defines the stochas- 
tic linear transformation known as the Karhunen-Loeve 
Transform [vllsst. 

" < — iTi — > " 



with m = N . Because such a transformation opti- 
mally decorrelates the activation patterns, concentrating 
the variance of the observations along the first axes (the 
so-called principal axes or variables), it is frequently pos- 
sible to reduce the dimensionality of the measurements 
without substantial loss of information by considering the 
above matrix with m <^ N. The new, projected measure- 
ments g, with dimension m, can now be straightforwardly 
obtained in terms of the following linear transformation 

g^Gf. (4) 



III. RESULTS AND DISCUSSION 

Figure [5] illustrates the 9 networks adopted in the 
present investigation. Each of them involves 4 communi- 
ties, of respective ER, BA, WS and GG types (see legend 
at the bottom of the figure) and approximately 50 nodes 
each. The intra-community degrees, expressed in terms 
of the BA parameter m, increase along the columns (top 
to bottom), and the inter-community degrees k increase 
along the rows (left to right). The considered values of 
intra- and inter-connectivity are shown in Figure [2l The 
same intra-connectivity degree, defined with respect to 
the parameter m of the BA model, was adopted for all 
the 4 communities in each case. The consideration of 
hybrid communities involving several network models is 
particularly useful for investigating the community de- 
tection methodology with respect to varying connectivity 
patterns. 

Each network was searched for community structure 
by placing the activation source (with intensity 1) at 
each of its neurons and simulating the respective acti- 
vation and spiking along the initial H = 200 steps of 
the transient dynamics. Three whole set of simulations 
where performed by considering respective decay rates a 
equal to 0.02 and 0.5. Figure [3] shows the activogram and 
spikegram, as well as the diagrams of beginning activa- 
tion times and beginning spiking times for the network 
with m = 3, fc = 0.2 and a = 0.02. 

The constant activation fed through neuron 25 is 
clearly identified as the white column in the activogram 
and spikegram. Because of the more intense intercon- 
nectivity between the nodes in the community to which 
this neuron belongs (ER, in this case) the propagation of 
activation and spikes tend to occur first inside this com- 
munity, being propagated to the other communities only 
later. One exception are the neurons around node 125, 
which belong to the WS community. Because neuron 
125 is connected to the ER community with particular 
intensity (more than one edge), it receives considerable 
activation sooner, leading to a progressive spreading of 
activation within the WS community. However, because 
this effect is not verified for most of the other nodes of 
the ER community, it tends to become less relevant in the 
subsequent decorrelation projection implemented by the 
PCA. In addition, except for a few other cells, the nodes 
which are inside the same community tend to receive ac- 
tivation relatively soon, as illustrated in the respective 
diagram of beginning activation times. A less regularly 
simultaneous activation is obtained for the spikes in the 
respective beginning spiking times diagram. The incor- 
poration of suitable (not too large) values of decay seems 
to promote more stable activation patterns for the source 
placed at neurons of a same community as a consequence 
of further constraints on the dispersion of the activation. 
In this work we consider for the community identifica- 
tion the patterns of activation obtained by integrating 
the activation from time to H = 200. Observe that the 
parameter H has important implications for the compu- 
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tational cost, in the sense that the larger its value, the 
larger the number of computations. 

Figure |4] depicts the clusters obtained in the two- 
dimensional space defined by the first two PC A variables 
considering the average activation patterns and decay 
a = 0.02. Figure [5] shows the respective scatterplots 
obtained for the first and third PCA variables. There- 
fore, it is possible to have a clear idea of the 3D PCA 
space by considering these two images. In these figures, 
as well as all the other subsequent ones, the original com- 
munities are identified by respective colors: ER in blue; 
BA in green; WS in red and GG in magenta. In most 
cases, especially for low ratios fc/m, the original commu- 
nities were mapped into well-defined respective clusters 
in the PCA space. For instance, in the case m — 2 and 

= 0.1, we have dense clusters obtained for the ER and 
BA communities. Because of their intrinsic nature, the 
WS and GG models tended to yield larger dispersions in 
most of the cases considered in this work. Yet, they are 
well-separated, as it can be verified by considering both 
the peal X pca2 (Figure H]) and peal x pea3 (Figure [S]) 
diagrams. Except for the WS case, the other 3 com- 
munities still tended to map to reasonably well-defined 
local regions in the PCA projections for larger values of 
intercommunity connection (i.e. fc — 0.5 and 1). The 
separation between the community clusters tended to in- 
crease substantially from top to down along each column 
in Figures |4] and [5] as a consequence of the increase of 
the intra-community connectivity relatively to the inter- 
community density of connections. 

The PCA scatterplots obtained by considering more 
intense decay (i.e. a = 0.5) are shown in Figures [6] and [7| 
respectively to the peal x pca2 and peal x pea3 projec- 
tions. Less separated clusters have been obtained in most 
cases, with intense overlap between communities. How- 
ever, the nodes belonging to the original communities still 
tended to be mapped to nearby positions in the scatter- 
plots. Such a decrease in the community identification is 
a direct consequence of the fact that more intense decays 
tended to produce less stable activation patterns for the 
activation source placed at different nodes. In addition, 
the consideration of more intense decay also would im- 
ply in averaging the activations along a longer period of 
time, demanding additional computations. 



IV. CONCLUDING REMARKS 

In continuation to recent previous investigations [SS^, 
the relationship between topological and dynamical mod- 
ularity during the transient period of activation of non- 
linear integrate-and-fire complex neuronal networks has 
been explored further, with respect to the consideration 
of average activation patterns as resources for community 
identification. By increasing the latent period in which 
the activation has to increase until firing is reached in 
the neuronal cells, relatively small non-zero decays of the 
accumulated activation tended to allow the subsequent 



identification of the topological communities as clusters 
appearing in scatterplots obtained by optimally decorre- 
lated PCA projections. 

Such a phenomenon has been experimentally investi- 
gated by considering several hybrid networks, with com- 
munities of different types, and varying ratios of intra- to 
intercommunity connectivity. In most cases the original 
communities were mapped onto adjacent sets of points 
(clusters) which were often well-defined and delimitated. 
As in f33l|, the nodes at the interfaces between the ob- 
tained clusters tended to correspond to those nodes im- 
plementing the intercommunity connectionsl. The ER 
and BA modules tended to produce more concentrated 
clusters, with the WS and GG communities often yield- 
ing scattered, but still separated, distributions in the 
PCA diagrams. The discrimination between the commu- 
nities was undermined when a substantially large decay 
was considered. As observed previously '33|, the tran- 
sient confinement of the activation inside each commu- 
nity seems to be related to an abrupt pattern of activa- 
tion observed for several models of complex networks [13] . 
As a matter of fact, the WS and GG models had indeed 
been observed [s^ to produce less abrupt activations. 
The situations involving higher values of a would also 
imply substantially higher computational cost required 
for the simulation of the activation along longer tran- 
sient periods. Nevertheless, it seems to follow from the 
currently reported results that relatively small non-zero 
decay of the accumulated activation, to a certain extent, 
emphasizes the transient confinement of activation inside 
the communities. 

These findings substantiate the importance of transient 
dynamics for the characterization and analysis of non- 
linear complex systems. Furthermore, the relationship 
between modular interconnectivity and nearly simultane- 
ous activation of communities has several implications for 
biological and computational neuroscience. In particular, 
such a relationship can be intrinsically related to recog- 
nition of patterns and associative memory. For instance, 
the nearly simultaneous activation of the communities 
could play an important role in reconstructing larger 
patterns (communities) from incomplete presentations. 
Because of the temporal dynamics of nervous systems, 
where problems have to be solved by functional modules 
of neurons along a given period of time (e.g. [H, H^l), it 
is possible that the phenomenon of simultaneous activa- 
tion within communities plays an important general role 
in neuronal organization and functionality. The neuronal 
community identification methodology is also promising 
for the identification of functional modules in the cortex 
or neuronal subsystems because of its intrinsic compat- 
ibility with the non-linear dynamics performed in those 
systems. 

Several are the future works implied by the results 
and methods reported in the current article. First, it 
would be important to perform more objective investiga- 
tions of the discriminability between the PCA clusters, 
for instance by using the intra- and inter-class scatterings 
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(e.g. j35l|) and comparing the results obtained for the syn- 
thetic hybrid communities with those yielded by canon- 
ical analysis (e.g. 0, [11]). It would also be necessary to 
consider larger ensemble of networks in order to reach 
more general and definitive conclusions regarding the ef- 
fect of the few parameters involved (i.e. a and H), as 
well as concerining possible finite-size and scaling effects. 
Valuable insights about the influence of the connectiv- 
ity on the transient non-linear dynamics of the complex 
neuronal networks considered in this work can be po- 
tentially achieved by applying the systematic approach 
of superedges [ll] . Of particular interest are further in- 
vestigations aimed at the characterization of the abrupt 



activations verified for several complex network models. 
This phenomenon, which is possibly associated to phase 
transition and/or self-organized criticality, seems to lie 
at the heart of the confinement of the activation inside 
the communities during the transient activation. 
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FIG. 2: The 9 hybrid networks considered in this work incorporate 4 communities each, of respective ER, BA, WS and GG 
types (see legend at the bottom). 
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FIG. 3: The activogram and spikegram, as well as the diagrams of beginning activation times and beginning spiking times 
obtained for the 200 initial time steps with activation source at node 25 for the complex network with m = 3, k — 0.5 (Figure[2^) 
and Q = 0.02. 
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FIG. 7: The clusters obtained by considering the first and third PCA variables for a = 0.5. 



